Scene Graph Generation with External Knowledge and Image Reconstruction

Scene graph generation has received growing attention with the advancements in image understanding tasks such as object detection, attributes and relationship prediction,~\etc. However, existing datasets are biased in terms of object and relationship labels, or often come with noisy and missing annotations, which makes the development of a reliable scene graph prediction model very challenging. In this paper, we propose a novel scene graph generation algorithm with external knowledge and image reconstruction loss to overcome these dataset issues. In particular, we extract commonsense knowledge from the external knowledge base to refine object and phrase features for improving generalizability in scene graph generation. To address the bias of noisy object annotations, we introduce an auxiliary image reconstruction path to regularize the scene graph generation network. Extensive experiments show that our framework can generate better scene graphs, achieving the state-of-the-art performance on two benchmark datasets: Visual Relationship Detection and Visual Genome datasets.Comment: 10 pages, 5 figures, Accepted in CVPR 201

Estimating the HARA Land Use Model for Housing Planning based on Hedonic Price Analysis

HARA is a land-use model that uses a search algorithm to find the optimal spatial allocation of new housing demands in an urban plan area. In the model, the plan area is represented as a grid of cells. A core element of the algorithm is a function that is used to evaluate the value of a cell for each possible land-use given its location. The value function is specified as the net value of a (housing) development given the land costs, the construction costs, and the market value of the development at a location. Specified in that way, the solution generated represents an optimum as well as a market equilibrium (maximum net value for developers). A critical prerequisite for this is, however, that the value-function is specified such that it accurately represents buyers’ willingness-to-pay for dwelling and location characteristics in the housing market. In this paper, we show how the value function can be estimated using hedonic price analysis. The analysis is carried out based on a large housing transaction data set focusing on two medium-sized cities in The Netherlands combined with detailed land-use data of these areas. Although a full set of land-use types is taken into account, special attention is paid to the classification of urban green space, given the purpose to analyze scenarios for developing urban green space. The results indicate that land-use effects on housing prices differ considerably between housing types as well as city. We conclude therefore that it is important in the estimation of land-use models to take the specific local conditions of housing markets and housing segments into account

Memory Efficient Optimizers with 4-bit States

Optimizer states are a major source of memory consumption for training neural networks, limiting the maximum trainable model within given memory budget. Compressing the optimizer states from 32-bit floating points to lower bitwidth is promising to reduce the training memory footprint, while the current lowest achievable bitwidth is 8-bit. In this work, we push optimizer states bitwidth down to 4-bit through a detailed empirical analysis of first and second moments. Specifically, we find that moments have complicated outlier patterns, that current block-wise quantization cannot accurately approximate. We use a smaller block size and propose to utilize both row-wise and column-wise information for better quantization. We further identify a zero point problem of quantizing the second moment, and solve this problem with a linear quantizer that excludes the zero point. Our 4-bit optimizer is evaluated on a wide variety of benchmarks including natural language understanding, machine translation, image classification, and instruction tuning. On all the tasks our optimizers can achieve comparable accuracy with their full-precision counterparts, while enjoying better memory efficiency.Comment: 35 page